Stochastic Problem Research Articles

Adaptive Lagrangian Policies for a Multiwarehouse, Multistore Inventory System with Lost Sales

Adaptive Lagrangian Policies for Inventory Control Consider the problem of managing inventory in a multiwarehouse, multistore setting where each store can be periodically replenished from potentially a set of nearby warehouses. This is a practically relevant problem, and its optimal policy is not easy to compute because of the curse of dimensionality. Existing works in the literature have proposed a policy based on Lagrangian relaxation of the original stochastic problem. This vanilla policy applies the control parameters derived from the Lagrangian model and has been shown to perform well in some cases. In “Adaptive Lagrangian Policies for a Multiwarehouse, Multistore Inventory System with Lost Sales,” we go one step farther and develop adaptive Lagrangian policies that update (at certain update times) the control parameters of the vanilla policy based on the historical demand realizations. We analytically show that our proposed adjustment significantly improves the performance of the vanilla policy.

A reinforcement learning framework for improving parking decisions in last-mile delivery

This study leverages simulation-optimisation with a Reinforcement Learning (RL) model to analyse the routing behaviour of delivery vehicles (DVs). We conceptualise the system as a stochastic k-armed bandit problem, representing a sequential interaction between a learner (the DV) and its surrounding environment. Each DV is assigned a random number of customers and an initial delivery route. If a loading zone is unavailable, the RL model is used to select a delivery strategy, thereby modifying its route accordingly. The penalty is gauged by the additional trucking and walking time incurred compared to the originally planned route. Our methodology is tested on a simulated network featuring realistic traffic conditions and a fleet of DVs employing four distinct lastmile delivery strategies. The results of our numerical experiments underscore the advantages of providing DVs with an RL-based decision support system for en-route decision-making, yielding benefits to the overall efficiency of the transport network. Highlights Combining simulation and optimisation algorithms with reinforcement learning Model DVs en-route parking decisions with a k-armed bandit algorithm Evaluating the impacts of delivery strategies on traffic congestion and in last-mile delivery efficiency

Comparative study of robust optimization model of multimaterial emergency logistics in an uncertain environment

AbstractGiven the significance of objective functions in alleviating human suffering within the humanitarian relief supply chain, it is imperative to consider both logistics costs and the cost of deprivation as ideal targets. This study investigates a multimaterial single‐cycle stochastic mixed integer model, integrating uncertainties about post‐disaster demand, supply survival, and road networks. An innovative heuristic approach is used to solve the stochastic facility location problem, which minimizes unfulfilled requests and response times in lexicographic order. This research also compares the cost‐minimization model of the pre‐positioning problem to an alternative model that maximizes demand satisfaction under resource constraints. Empirical validation through numerous case studies and solutions confirms the effectiveness and feasibility of the alternative model. The results of our experiments consistently demonstrate that the best solution provided by the alternative model not only satisfies the minimum cost model but also surpasses the conventional problem, without compromising performance. Consequently, it is crucial to avoid the cost‐minimization approach at the expense of human suffering.

Navigating uncertain distribution problem: a new approach for resolution optimization of transportation with several objectives under uncertainty

Amidst uncertainty, decision-making in manufacturing becomes a central focus due to its complexity. This study explores complex transportation constraints and uses novel ways to guide manufacturers. The Multi-objective Stochastic Linear Fractional Transportation Problem (MOSLFTP) is a crucial tool for managing supply chains, manufacturing operations, energy distribution, emergency routes, healthcare logistics, and other related areas. It adeptly addresses uncertainty, transforming efficiency and effectiveness in several domains. Stochastic programming is the process of converting theoretical probabilities into concrete certainties. The artistic compromise programming technique acts as a proficient mediator, reconciling opposing objectives and enabling equitable decision-making. This novel approach also addresses the Multi-objective Stochastic Linear plus Linear Fractional Transportation Problem (MOSLPLFTP), which involves two interconnected issues. The effectiveness of these principles is clearly shown with the help of the LINGO® 18 optimization solver. This study uses a ranking method to compare the similar methods to solve the current problems. A meticulously designed example acts as a significant achievement, shedding light on our method in a practical setting. It serves as a distinctive instrument, leading manufacturers through the maze of uncertainty and assisting them in determining the most advantageous course of action. This journey involves subtle interactions between complexity and simplicity, uncertainty is overcome by decisiveness, and invention is predominant.

A Stochastic Regression and Sentiment Analysis-Based Approach for Forecasting Trends in the Stock Market

Stochastic regression problems especially applied to time series forecasting problems often encounter the challenge of volatility and unpredictable seasonality in datasets. One such application happens to forecasting the movement of stock markets globally. Stock data across the world exhibits an intrinsic baseline noise, which often can’t be correlated to previous temporal patterns. Rather, such divergences are an effect of multiple non-numeric variables which govern stock movement such as political scenarios, trade wars, imminent economic waxing and waning trends to name a few. However, it is essential to incorporate these factors while training a model so as to design a strongly sentient and anticipatory regression algorithm which is both pervasive and robust, encompassing as many numeric and non-numeric factors as possible. While conventional machine learning and deep learning approaches tend to leverage tested model architectures, they often miss out on the optimization of both training data and training method to yield potentially high forecasting accuracy. The paper combines stochastic regression in terms of momentum based training and data optimization which is shown to exhibit considerable lower forecasting error compared to existing approaches. To encompass a wider feature space, sentiment analysis has been incorporated to correlate stock movements with public opinions. The proposed approach has been tested on a multitude of benchmark datasets, to validate the performance of the approach. The mean absolute percentage error (MAPE) and regression values have been selected as primary metrics for the performance evaluation.  The proposed approach attains a mean MAPE value of 3.22%. The analysis of the MAPE and forecasting accuracy w.r.t. existing benchmark approaches proves the improved forecasting performance of the proposed approach over a period of 300 days.

Analytical investigation of space–time shifted nonlocal stochastic Sasa–Satsuma equation using the enhanced modified extended tanh-expansion method for stochastic solitary waves Solutions

In this paper, discover exact solutions of the space–time shifted nonlocal stochastic Sasa–Satsuma equations using the enhanced modified extended tanh-expansion method. This method is a sophisticated mathematical approach for analytical and approximate solution of a nonlinear partial differential equations. The space–time shifted nonlocal stochastic Sasa–Satsuma equations define the behavior of two complex-valued functions, u and v, in a nonlinear system. These equations are complicated to solve analytically because they contain a number of components that describe space–time shifting, nonlocal interactions, and stochastic problems. To increase reliability and effectiveness of the solutions, this method couples the extended tanh-function method with an enhanced method. In order to demonstrate how white noise affects the solutions to the space–time shifted nonlocal stochastic Sasa–Satsuma equations, we also use Matlab to create 3D surfaces and contour graphs of exact solutions.

Minimax Optimal Bandits for Heavy Tail Rewards.

Stochastic multiarmed bandits (stochastic MABs) are a problem of sequential decision-making with noisy rewards, where an agent sequentially chooses actions under unknown reward distributions to minimize cumulative regret. The majority of prior works on stochastic MABs assume that the reward distribution of each action has bounded supports or follows light-tailed distribution, i.e., sub-Gaussian distribution. However, in a variety of decision-making problems, the reward distributions follow a heavy-tailed distribution. In this regard, we consider stochastic MABs with heavy-tailed rewards, whose p th moment is bounded by a constant νp for . First, we provide theoretical analysis on sub-optimality of the existing exploration methods for heavy-tailed rewards where it has been proven that existing exploration methods do not guarantee a minimax optimal regret bound. Second, to achieve the minimax optimality under heavy-tailed rewards, we propose a minimax optimal robust upper confidence bound (MR-UCB) by providing tight confidence bound of a p -robust estimator. Furthermore, we also propose a minimax optimal robust adaptively perturbed exploration (MR-APE) which is a randomized version of MR-UCB. In particular, unlike the existing robust exploration methods, both proposed methods have no dependence on νp . Third, we provide the gap-dependent and independent regret bounds of proposed methods and prove that both methods guarantee the minimax optimal regret bound for a heavy-tailed stochastic MAB problem. The proposed methods are the first algorithm that theoretically guarantees the minimax optimality under heavy-tailed reward settings to the best of our knowledge. Finally, we demonstrate the superiority of the proposed methods in simulation with Pareto and Fréchet noises with respect to regrets.

A sample average approximation-based heuristic for the stochastic production routing problem

AbstractThe Production Routing Problem under demand uncertainty is an integrated problem containing production, inventory, and distribution decisions. At the planning level, the aim is to meet retailers demand, when only the demand distribution is known in advance, while minimizing the corresponding costs. In this study, a two-stage formulation is presented in which the routing can be adjusted at short notice. In the first stage, only production decisions are made, while delivery and inventory quantities and retailer visit schedules are determined in the second stage. To handle a large number of scenarios, two solution methods based on Sample Average Approximation are introduced. Furthermore, the impact of the routing quality is explored by applying a simple heuristic and an effective metaheuristic on the routing part. It is shown that, on average, the simple heuristic within an adjustable Sample Average Approximation approach provides better objective function values than the metaheuristic within a non-adjustable approach. Also all solution approaches outperform an expected value based approach in terms of runtime and objective function value.

SOLVING STOCHASTIC TRANSPORTATION ELECTRICITY PROBLEM WITH FUZZY INFORMATION ON PROBABILITY DISTRIBUTION USING MATLAB PROGRAM

This study focuses on MATLAB code programs of the entire stages of solving Stochastic Transportation Linear Programming Problems with Fuzzy Uncertainty Information on Probability Distribution Space (STLPPFI) with its algorithm outlines. A MATLAB code program of STLPPFI problem solver with algorithm outlines are proposed to solve STLPPFI model problems, and it utilizes many concepts as Alpha-Cut technique, Truth Degrees technique, Linear Fuzzy Membership Function (LFMF), Trapezoidal Fuzzy Number , Triangular Fuzzy Number , Linear Fuzzy Ranking Function (LFRF), Expectation Weighted Summation technique (EWS) and analyzing cases via second condition test of alpha-cut technique. The STLPPFI problem solver is utlized to convert STLPPFI into its corresponding equivalent Deterministic Transportation Linear Programming Problem (DTLPP) via defuzzifying from fuzziness on probability distribution space and derandomization randomness of problem formulation respectively. In addition, Dual-Simplex algorithm method with Vogel Approximation Algorithm Method (VAM) are used to obtain optimal solution from DTLPP. All MATLAB code programs with their proposed algorithm outlines are new except Dual-Simplex and VAM. The MATLAB code program of STLPPFI problem solver are more efficient along with a numerical example on electricity field illustrating practicability of this proposed MATLAB code program with its algorithm. Finally, the solution procedure illustrates the MATLAB code program of the proposed method is practical and applicable in the fields of energy and industry as it facilitates the method of transforming the energy at the lowest cost, least time running and is commercially applicable. Comparative comments are provided between Dual-Simplex and VAM in solution process.

A Dependability Neural Network Approach for Short-Term Production Estimation of a Wind Power Plant

This paper presents a novel approach to estimating short-term production of wind farms, which are made up of numerous turbine generators. It harnesses the power of big data through a blend of data-driven and model-based methods. Specifically, it combines an Artificial Neural Network (ANN) for immediate future predictions of wind turbine power output with a stochastic model for dependability, using Hybrid Reliability Block Diagrams. A thorough state-of-the-art review has been conducted in order to demonstrate the applicability of an ANN for non-linear stochastic problems of energy or power forecast estimation. The study leverages an innovative cluster analysis to group wind turbines and reduce the computational effort of the ANN, with a dependability model that improves the accuracy of the data-driven output estimation. Therefore, the main novelty is the employment of a hybrid model that combines an ANN with a dependability stochastic model that accounts for the realistic operational scenarios of wind turbines, including their susceptibility to random shutdowns This approach marks a significant advancement in the field, introducing a methodology which can aid the design and the power production forecast. The research has been applied to a case study of a 24 MW wind farm located in the south of Italy, characterized by 28 turbines. The findings demonstrate that the integrated model significantly enhances short-term wind-energy production estimation, achieving a 480% improvement in accuracy over the solo-clustering approach.

Application of Stochastic Programming in Agricultural and Newsvendor Problems and It's Application in Real Life

The study of making the best decision under risk management in a variety of areas of our lives is known as Stochastic Programming. We will go through two-stage Stochastic Linear Programming approaches for a variety of real-world choice issues, as well as how to solve them. We will achieve this by constructing stochastic linear programming models based on real-world situations like the well-known Farmer's situation and News Vendors problems. The influence of pricing, Stochastic Integer Linear Programming problem, second stage Stochastic Integer Linear Programming problem, first stage Stochastic Binary Linear Programming problem, risk aversion problem, and continuous function for random variables based on two-stage SLP with the aid of Farmer's problem will all be examined. We will address the Newsvendor’s problem with Deterministic Equivalent Stochastic Linear Programming, an extension of Deterministic Stochastic Linear Programming for risk aversion with a high number of decision variables and restrictions, utilizing the two-stage Stochastic Linear Programming approach once more. Hand calculation is a challenging way to acquire the solution to the problems. As a result, we will use the programming language AMPL to design computer solutions for tackling both farmer and newsvendor difficulties. We will also utilize MATLAB to create graphs for the farmer's problem's continuous function. Dhaka Univ. J. Sci. 72(1): 30-45, 2024 (January)

No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learning

The large action space is one fundamental obstacle to deploying Reinforcement Learning methods in the real world. The numerous redundant actions will cause the agents to make repeated or invalid attempts, even leading to task failure. Although current algorithms conduct some initial explorations for this issue, they either suffer from rule-based systems or depend on expert demonstrations, which significantly limits their applicability in many real-world settings. In this work, we examine the theoretical analysis of what action can be eliminated in policy optimization and propose a novel redundant action filtering mechanism. Unlike other works, our method constructs the similarity factor by estimating the distance between the state distributions, which requires no prior knowledge. In addition, we combine the modified inverse model to avoid extensive computation in high-dimensional state space. We reveal the underlying structure of action spaces and propose a simple yet efficient redundant action filtering mechanism named No Prior Mask (NPM) based on the above techniques. We show the superior performance of our method by conducting extensive experiments on high-dimensional, pixel-input, and stochastic problems with various action redundancy tasks. Our code is public online at https://github.com/zhongdy15/npm.

Learning to Build Solutions in Stochastic Matching Problems Using Flows (Student Abstract)

Generative Flow Networks, known as GFlowNets, have been introduced in recent times, presenting an exciting possibility for neural networks to model distributions across various data structures. In this paper, we broaden their applicability to encompass scenarios where the data structures are optimal solutions of a combinatorial problem. Concretely, we propose the use of GFlowNets to learn the distribution of optimal solutions for kidney exchange problems (KEPs), a generalized form of matching problems involving cycles.

Hierarchical Planning and Learning for Robots in Stochastic Settings Using Zero-Shot Option Invention

This paper addresses the problem of inventing and using hierarchical representations for stochastic robot-planning problems. Rather than using hand-coded state or action representations as input, it presents new methods for learning how to create a high-level action representation for long-horizon, sparse reward robot planning problems in stochastic settings with unknown dynamics. After training, this system yields a robot-specific but environment independent planning system. Given new problem instances in unseen stochastic environments, it first creates zero-shot options (without any experience on the new environment) with dense pseudo-rewards and then uses them to solve the input problem in a hierarchical planning and refinement process. Theoretical results identify sufficient conditions for completeness of the presented approach. Extensive empirical analysis shows that even in settings that go beyond these sufficient conditions, this approach convincingly outperforms baselines by 2x in terms of solution time with orders of magnitude improvement in solution quality.

Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization

We present a new zero-order optimization method called Minibatch Stochastic Three Points (MiSTP), specifically designed to solve stochastic unconstrained minimization problems when only an approximate evaluation of the objective function is possible. MiSTP is an extension of the Stochastic Three Point Method (STP). The key innovation of MiSTP is that it selects the next point solely based on the objective function approximation, without relying on its exact evaluation. At each iteration, MiSTP generates a random search direction and compares the approximations of the objective function at the current point, the randomly generated direction and its opposite. The best of these three points is chosen as the next iterate. We analyze the worst-case complexity of MiSTP in the convex and non-convex cases and demonstrate that it matches the most accurate complexity bounds known in the literature for zero-order optimization methods. We perform extensive numerical evaluations to assess the computational efficiency of MiSTP and compare its performance to other state-of-the-art methods by testing it on several machine learning tasks. The results show that MiSTP outperforms or has comparable performance against state-of-the-art methods indicating its potential for a wide range of practical applications.

The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models

Thompson sampling (TS) has been known for its outstanding empirical performance supported by theoretical guarantees across various reward models in the classical stochastic multi-armed bandit problems. Nonetheless, its optimality is often restricted to specific priors due to the common observation that TS is fairly insensitive to the choice of the prior when it comes to asymptotic regret bounds. However, when the model contains multiple parameters, the optimality of TS highly depends on the choice of priors, which casts doubt on the generalizability of previous findings to other models. To address this gap, this study explores the impact of selecting noninformative priors, offering insights into the performance of TS when dealing with new models that lack theoretical understanding. We first extend the regret analysis of TS to the model of uniform distributions with unknown supports, which would be the simplest non-regular model. Our findings reveal that changing noninformative priors can significantly affect the expected regret, aligning with previously known results in other multiparameter bandit models. Although the uniform prior is shown to be optimal, we highlight the inherent limitation of its optimality, which is limited to specific parameterizations and emphasizes the significance of the invariance property of priors. In light of this limitation, we propose a slightly modified TS-based policy, called TS with Truncation (TS-T), which can achieve the asymptotic optimality for the Gaussian models and the uniform models by using the reference prior and the Jeffreys prior that are invariant under one-to-one reparameterizations. This policy provides an alternative approach to achieving optimality by employing fine-tuned truncation, which would be much easier than hunting for optimal priors in practice.

Efficient Constraint Generation for Stochastic Shortest Path Problems

Current methods for solving Stochastic Shortest Path Problems (SSPs) find states’ costs-to-go by applying Bellman backups, where state-of-the-art methods employ heuristics to select states to back up and prune. A fundamental limitation of these algorithms is their need to compute the cost-to-go for every applicable action during each state backup, leading to unnecessary computation for actions identified as sub-optimal. We present new connections between planning and operations research and, using this framework, we address this issue of unnecessary computation by introducing an efficient version of constraint generation for SSPs. This technique allows algorithms to ignore sub-optimal actions and avoid computing their costs-to-go. We also apply our novel technique to iLAO* resulting in a new algorithm, CG-iLAO*. Our experiments show that CG-iLAO* ignores up to 57% of iLAO*’s actions and it solves problems up to 8x and 3x faster than LRTDP and iLAO*.

Jointly Improving the Sample and Communication Complexities in Decentralized Stochastic Minimax Optimization

We propose a novel single-loop decentralized algorithm, DGDA-VR, for solving the stochastic nonconvex strongly-concave minimax problems over a connected network of agents, which are equipped with stochastic first-order oracles to estimate their local gradients. DGDA-VR, incorporating variance reduction, achieves O(ε^−3) oracle complexity and O(ε^−2) communication complexity without resorting to multi-communication rounds – both are optimal, i.e., matching the lower bounds for this class of problems. Since DGDA-VR does not require multiple communication rounds, it is applicable to a broader range of decentralized computational environments. To the best of our knowledge, this is the first distributed method using a single communication round in each iteration to jointly optimize the oracle and communication complexities for the problem considered here.

A Forward–Backward Simheuristic for the Stochastic Capacitated Dispersion Problem

In an effort to balance the distribution of services across a given territory, dispersion and diversity models typically aim to maximize the minimum distance between any pair of facilities. Specifically, in the capacitated dispersion problem (CDP), each facility has an associated capacity or level of service, and the objective is to select a set of facilities so that the minimum distance between any pair of them (dispersion) is maximized, while ensuring a user-defined level of service. This problem can be formulated as a linear integer model, where the sum of the capacities of the selected facilities must match or exceed the total demand in the network. Real-life applications often necessitate considering the levels of uncertainty affecting the capacity of the nodes. Failure to account for this uncertainty could lead to low-quality or infeasible solutions in practical scenarios. However, research addressing the stochastic version of the CDP is scarce. This paper introduces two models for the CDP with stochastic capacities, incorporating soft constraints and penalty costs for violating the total capacity constraint. The first model includes a probabilistic constraint to ensure the required level of service with a certain probability, while the second model introduces a soft constraint with penalty costs for violations. To solve both variants of the model, a forward–backward simheuristic algorithm is proposed. Our approach combines a metaheuristic algorithm with Monte Carlo simulation, enabling the efficient handling of the random behavior of node capacities and obtaining reliable solutions regardless of their probability distribution.

Designing Sparse Graphs for Stochastic Matching with an Application to Middle-Mile Transportation Management

Given an input graph [Formula: see text], we consider the problem of designing a sparse subgraph [Formula: see text] with [Formula: see text] that supports a large matching after some nodes in V are randomly deleted. We study four families of sparse graph designs (namely, clusters, rings, chains, and Erdős–Rényi graphs) and show both theoretically and numerically that their performance is close to the optimal one achieved by a complete graph. Our interest in the stochastic sparse graph design problem is primarily motivated by a collaboration with a leading e-commerce retailer in the context of its middle-mile delivery operations. We test our theoretical results using real data from our industry partner and conclude that adding a little flexibility to the routing network can significantly reduce transportation costs. This paper was accepted by David Simchi-Levi, optimization. Funding: This work was supported by the University of Chicago Booth School of Business, an Alibaba Cainiao Research Grant, and the Singapore Ministry of Education [NUS Startup Grant WBS A-0003856-00-00]. Supplemental Material: Data and the online appendix are available at https://doi.org/10.1287/mnsc.2022.01588 .